Numerical Analysis of Generalized Semi-Markov Processes
نویسندگان
چکیده
This paper present methodological results that allow the cost-effective numerical analysis of finite-state generalized semi-Markov processes (GSMPs) with exponential and deterministic events by an embedded general state space Markov chain (GSSMC). Key contributions constitute the formal proof that elements of the transition kernel of the GSSMC can always be computed by appropriate summation of transient state probabilities of continuous-time Markov chains and the derivation of conditions under which kernel elements are constant. Furthermore, we derive conditions on the building blocks of the GSMP for which state probabilities πi a a 1 2 , 1 6 are symmetric in respect to clock readings of deterministic events concurrently active. The exploitation of these properties is the key driver to the cost-effective time-dependent and stationary analysis of the considered class of GSMPs. The techniques of this paper are applicable to networks of queues, stochastic Petri nets, time-enhanced state charts and UML specifications, and other discrete-event stochastic systems with an underlying stochastic process that can be represented as a GSMP with exponential and deterministic events.
منابع مشابه
Continuity of Generalized Semi-Markov Processes
It is shown that sequences of generalized semi-Markov processes converge in the sense of weak convergence of random functions if associated sequences of defining elements (initial distributions, transition functions and clock time distributions) converge. This continuity or stability is used to obtain information about invariant probability measures. It is shown that there exists an invariant p...
متن کاملMulti-Grid Methods for Reinforcement Learning in Controlled Diffusion Processes
Reinforcement learning methods for discrete and semi-Markov decision problems such as Real-Time Dynamic Programming can be generalized for Controlled Diffusion Processes. The optimal control problem reduces to a boundary value problem for a fully nonlinear second-order elliptic differential equation of HamiltonJacobi-Bellman (HJB-) type. Numerical analysis provides multigrid methods for this ki...
متن کاملDistributed computation of transient state distributions and passage time quantiles in large semi-Markov models
Semi-Markov processes (SMPs) are expressive tools for modelling parallel and distributed systems; they are a generalisation of Markov processes that allow for arbitrarily distributed sojourn times. This paper presents an iterative technique for transient and passage time analysis of large structurally unrestricted semi-Markov processes. Our method is based on the calculation and subsequent nume...
متن کاملDistributed Computation of Passage Time Quantiles and Transient State Distributions in Large Semi-Markov Models
Semi-Markov processes (SMPs) are expressive tools for modelling concurrent systems; they are a generalisation of Markov processes that allow for arbitrarily distributed sojourn times. This paper presents an iterative technique for passage time and transient analysis of large structurally unrestricted semi-Markov processes. Our method is based on the calculation and subsequent numerical inversio...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1999